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@ Autonomous prioritized image transmission 

(^t) An image transmission system and method 
automatically to reduce the size of digital data 
signals transmitted from visual sensors. The 
system and method are completely automated 
and (a) use a plurality of sensors to detect 
targets in an area of interest, (b) direct a camera 
to place a 5 target in the camera's field of view, 
(c) provide a digitized visual image in which the 
target is Identified in a target box, and (d) 
automatically compress (or otherwise reduce 
the size of) a digital signal for nontarget por- 
tions of the visual image. 
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The present invention relates video transmission 
systems, and in which video information is automati- 
cally selected for transmission. 

Many automated security and observation sys- 
tems include video cameras that view an area of in- 
terest. Information from the video camera may be pro- 
vided real-time via conventional television communi- 
cation links to a display for monitoring by a human ob- 
server, and for storage on tape that may be replayed 
later. The human observer can identify potential tar- 
gets (a "target'* being any subject that is desirably de- 
tected by the system) and manually train the camera 
for closer observation. In one such system, the rail- 
road car identification system is disclosed in U.S. Pa- 
tent Specification No. 4,876,597. 

In some situations, conventional television com- 
munications links for transmitting video information 
are not adequate. For example, it may be desirable to 
send video data via a radio transmission system with 
a bandwidth too narrow to accommodate the desired 
transmission speeds (e.g., high frequency (HF) radio 
bandwidth is too narrow for conventional real-time tel- 
evision signals.) One solution to this problem is to 
transmit less information by eliminating some of the 
video data. As disclosed in U.S. Patent Specification 
No. 5,128,776, an operator may selectively designate 
important portions of the video data to be transmitted 
with no or visually lossless compression, and other 
less significant portions of the data with higher com- 
pression. However, human observers are still needed 
to identify the Important portions of the data and the 
camera field of view. 

Where portions of a video image appear consis- 
tently and are of predetermined appearance and as- 
pect, such as in a video telephone that displays a hu- 
man head facing the camera, a processor may locate 
this predetermined portion of the video image. When 
the head has been located and a boundary for the vid- 
eo data representing the head has been identified, 
the processor may reduce the amount of video data 
being transmitted by reducing the amount of data 
transmitted for portions outside the boundary relative 
to portions inside the boundary. (See the specifica- 
tion of U.S. Patent No. 4,951,140.) However, such a 
system will only detect targets having the predeter- 
mined appearance and aspect. 

Movement may also be used to reduce the 
amount of image data being transmitted. That is, it is 
known to use moving object detectors to identify 
movement by comparing sequential video images. 
Assuming that the moving portions are of interest, the 
amount of data for portions of the video image that are 
stationary may be reduced. (See the specification of 
the U.S. Patent No. 4,692,806.) In such systems, the 
moving object detector "tracks" the moving object (in 
other words, the least compressed portion of image 
data moves with the moving object). As is apparent, 
however, image data for an important portion of the 



image that may not be moving, or that is not moving 
enough to trigger the moving object deteqtor, may be 
compressed; for example, important image data for a 
target may be inadvertently compressed when the 

5 target is headed directly for the camera. 

An object of the present invention to provide an 
image transmission system and method of transmit- 
ting images and to provide a method and video image 
compression system that automatically detects a tar- 
to get in an area of interest and transmits a digital video 
image of the target in which nontarget information in 
the area of interest is automatically compressed be- 
fore transmission. 

The present invention includes a video image 

15 compression system for automatically detecting a tar- 
get in an area of interest and transmitting a digital im- 
age of the target wherein a digital image of nontarget 
information in the area of interest is automatically re- 
duced in size before transmission thereof comprising 

20 a target detector for searching an area of interest, said 
target detector detecting at least one of an acoustic, 
. seismic, infrared, electromagnetic, trip-wire, and vis- 
ual signal related to potential targets, a system con- 
troller for evaluating whether a signal from potential 

25 targets detected by said target detector is related to 
a target, said system controller comprising, i) means 
for evaluating signals from potential targets to deter- 
mine whether such signals identify a target, ii) means 
for determining a direction to an identified target, iii) 

30 means for using the determined direction to establish 
a target box within which the identified target is locat- 
ed, and iv) means for following target movement and 
retaining the target in said target box, a camera sys- 
tem that receives commands from said system con- 

35 troller for moving said camera system to said target 
box so that said camera system has said target box 
in a field of view, said camera system including, a) 
means for automatically adjusting the field of view 
and focus of said camera system so that an image of 

40 said target box and a portion of an adjacent area of 
interest can be generated by said camera system, and 
b) means for generating a digital signal representing 
said target box and said adjacent area of interest, said 
system controller further comprising, c) means for re- 

45 ducing the size of said digital signal of the adjacent 
area of interest that is to be transmitted, d) means for 
providing instructions for moving said camera system 
as said target box moves so that said target box is 
maintained in said camera system field of view, and 

50 e) means for designating a new adjacent area of in- 
terest uncovered by movement of said target box for 
transmission; and means for transmitting on a radio 
frequency said digital signal of said target box and 
said reduced size digital signal of said adjacent area 

55 of interest. 

The invention also includes a method of automat- 
ically detecting a target in an area of interest and 
transmitting a digital image of the target wherein a 
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digital image of nontarget information in the area of in- 
terest is reduced in size before transmission thereof, 
the method comprising the steps of: 

a) searching an area of interest with an automat- 
ed target detector, the target detector detecting at 5 
least one of an acoustic, seismic, infrared, elec- 
tromagnetic, trip-wire, and visual signal related to 

a potential target; 

b) upon detection of signal related to a potential 
target by the target detector, evaluating the de- io 
tected signal in a system controller to determine 
whether the potential target is a target; 

c) upon determination that the potential target is 
a target, determining in the system controller a di- 
rection to the target; is 

d) the system controller using the determined di- 
rection to establish a target box within which the 
target is located and thereafter retained by follow- 
ing target movement; 

e) providing a signal from the system controller to 20 
a camera system for moving the camera system 

so that the camera system has the target box in 
its field of view; 

f) automatically adjusting the field of view and fo- 
cus of the camera system so that an image of the 25 
target box and a portion of the adjacent area of 
interest may be generated by the camera system; 

g) generating a digital signal of the image of the 
target box and adjacent area of interest with the 
camera system; 30 

h) reducing the size of the digital signal of the ad- 
jacent area of interest; 

i) transmitting the digital signal of the target box 
and the reduced size signal of the adjacent area 

of interest; 35 
j) providing signals from the system controller for 
moving the field of view of the camera system as 
the target box moves so that the target box is 
maintained in the camera system field of view; 
and 40 
k) transmitting the digital signal of a new adjacent 
area of interest uncovered by the movement of 
the target box. 

The invention will now be described, by way of ex- 
ample, with reference to the accompanying drawings 45 
in which: 

Figure 1 is a block diagram of an embodiment of 
the video image transmission system. 

Figures 2a-2d are a series of images illustrating 
the sequence of operation. 50 

Figure 3 is a block diagram of a preferred embodi- 
ment of the present invention. 

In the system and method of the present inven- 
tion, the size of digital data signals transmitted from 
visual sensors are automatically reduced. The system 55 
and method are desirably completely automated and 
(a) use a plurality of sensors to detect targets in an 
area of interest, (b) direct a camera to place a target 



in the camera's field of view, (c) provide a digitized 
visual image in which the target is identified in a tar- 
get box, (d) automatically compress (or otherwise re- 
duce the size of) a digital signal for nontarget portions 
of the visual image, and (e) transmit background im- 
age data only once (i.e., once background data is 
sent, it is not re-sent unless a new camera field of 
view is required). 

With reference to Figure 1 and to Figures 2a-2d, 
the present invention includes a target detector 12 for 
locating potential targets PTi ^ in an area of interest 
Ai (Figure 2a). The target detector 12 may include at 
least one, and desirably several , sensors 1 2i , „ for de- 
tecting and providing a direction to potential targets. 
For example, the sensors may be acoustic, seismic, 
infrared, electromagnetic, trip-wire, visual, and the 
like. 

Information from the target detector 12 is provid- 
ed to a system controller 14 for evaluation. The sys- 
tem controller 14 determines whether a potential tar- 
get PTj is a target T based on all information available 
to it. The system controller 14 evaluates and corre- 
lates information from the sensors to screen potential 
targets PT^. in an area of interest A^. 

If a target T is identified, the system controller 14 
determines the location of the target T relative to a 
video camera 1 6 and provides commands to the video 
camera 16 so that the target T is placed in the field of 
view A2 (Figure 2b) of the video camera 16. To this 
end, the system controller 14 may determine a bear- 
ing and azimuth of the target and define a target box 
B in which the target is located. A target box B may 
take any shape and size and need not be a square as 
illustrated in the figures. For example, target box B 
may be the periphery of the target T. The system con- 
troller 14 coordinates movement of the video camera 
1 6 to place the target box B in the video camera's field 
of view A2. 

The video camera 16 is automatically adjusted so 
that its field of view and focus provide a useable view 
A3 of the target box B, such as illustrated in Figure 2c. 
Desirably the view A3 includes, in addition to the tar- 
get box B, some of the area of interest Ai adjacent the 
target box B to provide a view of the environment 
around the target T This result may also be achieved 
by enlarging the area around the target T that is en- 
compassed by the target box B. The video camera 16 
generates a signal representing the image A3 that 
may be digital or converted to digital in an analog-to- 
digital converter 18. The system controller 14 may 
use the digital signals from the A/D converter 18 (or 
signals directly from the video camera) to coordinate 
movement of the video camera 16 to place the target 
box B in its field of view. 

The digital signals are provided from the system 
controller 14 to a data reduction unit 20 that operates 
under the control of the system controller to reduce 
the amount of data representing the image A3 that will 
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be forwarded to a radio frequency transmitter 22 or 
other narrow band communications systems for 
transmission. The reduction unit 20 may selectively 
compress portions of the data, reduce the resolution 
of portions of the image to be transmitted, and/or not 
forward portions of the data to the transmitter 22 at 
all or only periodically or only when the target moves 
outside the current field of view of the video camera 
16. Desirably the data reduction unit 20 reduces the 
amount of data representing portions of the video im- 
age outside of the target box B, although data repre- 
senting the target box B may also be reduced to a 
lesser extent. For example, data representing the tar- 
get box B may be compressed so that no image data 
is lost and may be sent without interruption, while the 
data representing the adjacent area may be com- 
pressed and sent only once the first time it comes into 
the video camera field of view. The extent of data re- 
duction may be preselected, depending in part on the 
number and nature of the targets expected to be de- 
tected and the search environment. For example, 
where readily identifiable targets are to be searched 
in a relatively unchanging background ( e.g. , ships at 
sea) the background may be sent only once and/or 
highly compressed, while in a more complex back- 
ground such as a city street the background may be 
sent more frequently and/or compressed less. 

The target T may move, and when it does, the tar- 
get box B moves with it. Target movement may be de- 
termined using video camera 16 signals alone or in 
combination with information from the target detector 
12. With reference to Figure 2d. as the target box B 
moves, the system controller 14 coordinates move- 
ment of the video camera 16 so that the target box B 
remains in the camera's new field of view A4. 

In Figure 2d when the target T moves it will un- 
cover a previously untransmitted portion of the area 
of interest. This portion is new adjacent area N and 
data representing the area N may be forwarded to the 
transmitter after determination that it does not con- 
tain a new target and appropriate data reduction. The 
data representing the area N may be forwarded in 
turn or at once regardless of the status of transmis- 
sion of data representing the remaining adjacent area 
of interest. Once forwarded, data for the new adjacent 
area N is treated the same as data from the remaining 
adjacent area (e.g., periodically or not forwarded 
again, compressed, resolution reduced). 

Some targets may move enough to first cover 
and then uncover an adjacent area of interest for 
which data has already been transmitted. In this 
event, data for such adjacent areas need not be re- 
transmitted. A receiving unit would recognize this (or 
receive a signal from the transmitting unit so indicat- 
ing) and reconstruct the area from the previously 
transmitted data. Appropriate data storage facilities 
may be provided. 

Plural targets may be tracked and new targets 



BNSDOCID: <EP_0689357A1J_> 




689 357 A1 6 

may be added as they are detected, or uncovered 
when a previously designated target moves. 

The target detector 1 2 includes at least one of an 
acoustic, seismic, infrared, electromagnetic, trip- 

5 wire, and visual sensors 12i that can detect and 
provide an indication of direction to potential targets. 
Such sensors are well known and need not be dis- 
cussed herein. In one embodiment of the present in- 
vention, the target detector 12 includes several sen- 

10 sors 12i. .n spaced from one another so that range to 
the target can be determined, although this is not re- 
quired for the present invention. Sensors of diverse 
types are desirable so that target information can be 
correlated to evaluate and screen target types. For 

15 example, if only cars are to tracked, potential targets 
that do not have the appropriate infrared, or acoustic 
signal may be ignored. 

The system controller 14 and data reduction unit 
20 may include components known in the art for per- 

20 forming the various functions discussed above. For 
example, the patents discussed in the Background of 
the Invention include components for designating 
and tracking a moving object, for reducing image 
data, and for converting signals from a video camera 

25 into digital signals. Ameans for designating and track- 
ing a moving object is also available from Alliant Tech- 
systems of 600 Second Street, NE, Hopkins, Minne- 
sota, and is used in a preferred embodiment. 

The system controller 14 may include a conven- 

30 tional processor and data storage for evaluating the 
potential targets and for coordinating its various com- 
ponents. Characteristics of potential targets may be 
stored therein so that signals from the sensors 12^ 
may be evaluated. The controller 14 may also eval- 

35 uate the periodicity of features that move, but do not 
correlate to signals from sensors 12i...n- For example, 
moving trees or water may be indicated as potential 
targets by movement alone, and yet it is desirable to 
reduce the amount of data for such features. Accord- 

40 ingly, where features move in a repeated pattern and 
do not correlate to other sensors, the system control- 
ler 14 may direct the data reduction unit 20 to reduce 
data for these portions of the image. 

The video camera 1 6 may be a conventional cam- 

45 era for providing signals related to moving images, 
such as a television camera, a commercial video cam- 
era and the like. The analog-to-digital converter 18 
and radio frequency transmitter 22 may also be con- 
ventional. 

50 Figure 3 may include conventional acoustic sen- 

sors 30 and infrared sensors 32 for detecting poten- 
tial targets. Each type of sensor may have its own 
processors 34, 36 for providing information about the 
potential targets to system controller 38. The acoustic 

55 sensors 30 provide information about prominent 
sound frequencies being received and directions to 
each, while the infrared sensors 32 indicate that sig- 
nals that exceed a preselected strength have been re- 

4 
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ceived. 

Upon indication that a potential target has been 
found, the controller 38 may provide instructions 
(e.g., pan, tilt, zoom, focus) to a camera system 40 
and the infrared sensors 32 to move them to the bear- 5 
ings indicated by the acoustic sensors for each of the 
potential targets. The camera system 40 and infrared 
sensors 32 provide images of the potential targets 
that are evaluated by the controller 38, along with in- 
formation from the acoustic sensors 30, to determine io 
whether they are targets. For example, if a potential 
target emits an acoustic signature that is compatible 
with signatures for known targets (stored In the con- 
troller 38), and the camera and infrared sensors pro- 
vide corroborating Information, the potential target is 
may be identified as a target. The controller 38 eval- 
uates successive frames of image data to reject false 
alarms and to classify targets. 

Once a target is designated, the controller 38 
tracks the image of the target (the moving target box 20 
discussed above) and provides appropriate move- 
ment signals to the camera system 40 to keep the tar- 
get in the field of view. The infrared sensors 32 may 
be returned to a detection mode. 

The video signals from the camera system 40 are 25 
provided to a compression and transmission proces- 
sor 4 2 for preparation for transmission. In response to 
instructions from the controller 38, the processor 42 
reduces the amount of data representing the video 
signal in the manner discussed above (compression, 30 
resolution, transmission frequency) and provides the 
data representing the reduced size video signal to a 
transmitter 44. 

Multiple targets may be identified and tracked by 
using multiple cameras, by providing a camera field 35 
of view large enough to encompass all targets, or by 
panning a single camera periodically to each target. 

An image transmission system and method auto- 
matically to reduce the size of digital data signals 
transmitted from visual sensors. The system and 40 2. 
method are completely automated and (a) use a plur- 
ality of sensors to detect targets in an area of interest, 
(b) direct a camera to place a target in the camera's 
field of view, (c) provide a digitized visual image In 
which the target is identified in a target box, and (d). 45 
automatically compress (or otherwise reduce the size 
oO a digital signal for nontarget portions of the visual 
image. 3. 



Claims 

1. A video image compression system for automat- 
ically detecting a target In an area of Interest and 
transmitting a digital image of the target in which 55 
a digital image of nontarget information in the 4. 
area of interest is automatically reduced in size 
before transmission thereof comprising a target 
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detector for searching an area of interest, said tar- 
get detector detecting at least one of an acoustic, 
seismic, infrared, electromagnetic, trip-wire, and 
visual signal related to potential targets, a system 
controller for evaluating whether a signal from 
potential targets detected by said target detector 
is related to a target, said system controller com- 
prising, i) means for evaluating signals from po- 
tential targets to determine whether such signals 
identify a target, ii) means for determining a di- 
rection to an identified target, iil) means for using 
the determined direction to establish a target box 
within which the identified target is located, and 
Iv) means for following target movement and re- 
taining the target in said target box, a camera sys- 
tem that receives commands from said system 
controller for moving said camera system to said 
target box so that said camera system has said 
target box in a field of view, said camera system 
including, a) means for automatically adjusting 
the field of view and focus of said camera system 
so that an image of said target box and a portion 
of an adjacent area of interest can be generated 
by said camera system, and b) means for gener- 



ating a digital signal representing said target box -r *rm 

and said adjacent area of interest, said system . M 

controller further comprising, c) means for reduc- " 
Ing the size of said digital signal of the adjacent . .::m 

area of interest that is to be transmitted,' d) means 
for providing instructions for moving said camera t 
system as said target box moves so that said tar- 
get box is maintained in said camera system field ^ y-t 

of view, and e) means for designating a new ad- a 
jacent area of interest uncovered by movement of t 
said target box for transmission; and means for ^ 
transmitting on a radio frequency said digital sig- ; r.^ - 

nal of said target box and said reduced size digital • 
signal of said adjacent area of interest. 



An image compression system as claimed in 
Claim 1 wherein said means for reducing the size 
of said digital signal comprises means for trans- 
mitting said digital signal of said adjacent area of 
interest periodically, and of interest when it first 
appears in said camera system field of view and 
not thereafter. 

An Image compression system as claimed in 
Claim 1 or 2 wherein said means for following tar- 
get movement comprises means for evaluating 
the periodicity of moving portions of the digital im- 
age, and preferably further comprising means for 
compressing said digital signal representing said 
target box. 

An image compression system as claimed in any 
of Claims 1 to 3 wherein said target detector com- 
prises an acoustic sensor and an infrared sensor. 
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5. An image compression system as claimed in 
Claim 5 wherein said camera system comprises 
a video camera, and said means for evaluating 
potential targets comprises means for evaluating 
signals from said video camera, said acoustic s 
sensor and said infrared sensor. 

6. A method of automatically detecting a target in an 
area of interest and transmitting a digital image 

of the target wherein a digital image of nontarget io 
information in the area of interest is reduced in 
size before transmission thereof, the method 
comprising the steps of: 

a) searching an area of interest with an auto- 
mated target detector, the target detector de- is 
tecting at least one of an acoustic, seismic, in- 
frared, electromagnetic, trip-wire, and visual 
signal related to a potential target; 

b) upon detection of signal related to a poten- 
tial target by the target detector, evaluating 20 
the detected signal in a system controller to 
determine whether the potential target is a tar- 
get; 

. c) upon determination that the potential target 
is a target, determining in the system control- 25 
ler a direction to the target; 

d) the system controller using the determined 
direction to establish a target box within which 
the target is located and thereafter retained by 
following target movement; 30 

e) providing a signal from the system control- 
ler to a camera system for moving the camera 
system so that the camera system has the tar- 
get box in its field of view; 

f) automatically adjusting the field of view and 35 
focus of the camera system so that an image 

of the target box and a portion of the adjacent 
area of interest may be generated by the cam- 
era system; 

g) generating a digital signal of the image of 40 
the target box and adjacent area of interest 

with the camera system; 

h) reducing the size of the digital signal of the 
adjacent area of interest; 

i) transmitting the digital signal of the target 45 
box and the reduced size signal of the adja- 
cent area of interest; 

j) providing signals from the system controller 
for moving the field of view of the camera sys- 
tem as the target box moves so that the target so 
box is maintained in the camera system field 
of view; and 

k) transmitting the digital signal of a new ad- 
jacent area of interest uncovered by the move- 
ment of the target box. 55 

7. A method as claimed in Claim 6 wherein multiple 
targets are detected and separate target boxes 



are established for each of the multiple targets, 
and the digital signal of the adjacent area of inter- 
est is transmitted periodically. 

8. A method as claimed in Claim 7 wherein the dig- 
ital signal of the adjacent area of interest is trans- 
mitted only the first time it appears in the camera 
system field of view. 

9. A method as claimed in Claims 7 or 8 wherein 
when movement of the target box uncovers a new 
adjacent area of interest, the digital signal of the 
uncovered adjacent area of interest is transmit- 
ted immediately, and preferably the digital signal 
of the adjacent area of interest is compressed to 
a predetermined first level. 

10. A method as claimed in Claim 9 wherein the dig- 
ital image of the target box is compressed to a 
second level less than the first level, and in which 
the digital signal of the adjacent area of interest 
is transmitted only when a new adjacent area is 
required because of target movement. 



BNSDOCID: <EP 0689357 A1 I > 



EP 0 689 357 A1 




14- 



FIG, 1 



CONTROLLER 

I 




PI, - PT, 



m 









^ — 

' 1 
1 

I 







FIG. 2 a 



I 
I 
I 



B )l 



FIG. 2b 







FIG. 



2 c 



/ 



B ^4 



/ 

r - ^ - r - 



.0689357 A1J_> 



EP 0 689 357 A1 



>f 



,--'40 



CAMERA 
SYSTEMS 



VIDEO SIGNAL 



IMAGE COMPRESSION 
AND TRANSMISSION 
PROCESSORS 



,-42 

' , IMAGE 
DATA 



,-44 



38 



PAN/TILT, ZOOM AND 
FOCUS CONTROL 



30 



COMMUNICATION 
EQUIPMENT 



COMMAND DATA AND TARGET INFO 



^ IMAGE TARGET DETECTION 
AND TRACKING PROCESSORS 



ACOUSTIC 
SENSORS 




~ 




IR SENSORS 







34 



ACOUSTIC TARGET DETECTION 
AND TRACKING PROCESSORS 



IR TARGET DETECTION AND 
TRACKING PROCESSORS ■ 



32 



36-" 

FIG. 3 



POTENTIAL 

TARGET 

DATA 



8 



EP 0 689 357 A1 




European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 9S 30 4056 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indic«6on, where appropriate, 
of relevant passag<s 



FR-A-2 693 868 (HYMATOM SA) 21 January 
1994 

* page 2, line 7 - page 8, line 23 * 

* figures 1-6 * 

WO-A-90 01706 (HUGHES AIRCRAFT COMPANY) 22 
February 1990 

* page 2, line 18 - line 36 * 

* page 3, line 16 - page 11, line 32 * 

* figures 1-3 * 

US-A-4 961 211 (TSUGANE ET AL) 2 October 
1990 

* column 3, line 26 - column 9, line 24 * 

* figures 1-10 * 



Relevant 
to claim 



1,6 



1-10 



1-10 



CLASSIFICATION OF THE 
APPUCAnON OnLa.6) 



H04N7/18 
G01S3/786 



TECHNICAL FIELDS 
SEARCHF-D (lat.a.6) 



H04N 
GO IS 



The present sewch report has been drawn up for all ctaims 



Place «f Mnk 

THE HAGUE 



Diie mf mm^yam of tte Mvrfe 

28 September 1995 



Van der Zaal , R 



CATEGORY OF CITED DOCUMENTS 

X : particulairly releruit if taken alone 

Y : particularly relevant if combined with another 

document of the same caleKory 
A : technological backfiround 
O : Ron-wrivt<n disclosure 
P : intermediate document 



T : theory or prindple anderlylBg the Invention 
E : earlier patoit document, bat pablisKed on, or 

after the filing date 
D : documoit died in the application 
L : document dt«d for other reasons 

A : member of the same patent family, correspond hig 
document 



BNSDOCID: <EP_0689357A1 J_> 



THIS FA@i mm 



